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Volume I: Technical Consultation Report 
1.0 Authorization and Notification 

The request to conduct a real-time consultation involving the Space Transportation System 
(STS)-l 14 Space Shuttle Main Engine (SSME) Engine Cut-off (ECO) sensor anomaly was 
submitted to the NASA Engineering and Safety Center (NESC) on July 14, 2005. 

The authority to participate in a real-time consultation was approved by the NESC Review Board 
(NRB) in an out-of-board action on July 14, 2005. 

A final report will be presented to the NRB on November 3, 2005. 
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2.0 Signature Page 


Consultation Team Members 


Timmy R. Wilson, KSC, Lead Robert Kichak, GSFC 

NESC Chief Engineer NESC Discipline Expert for Avionics 


Robert Chemey, GSFC Eugene Ungar, JSC 

Orbital Sciences Corporation Deputy NESC Discipline Expert for Fluids 


Steve Rickman, JSC 
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3.0 Team Members, Ex Officio Members, and Consultants 


Timmy R. Wilson, KSC 
Robert Kichak, GSFC 
Eugene Ungar, JSC 
Robert Chemey, GSFC 
Steve Rickman, JSC 


NESC Chief Engineer (NCE) 

NESC Discipline Engineer (NDE) for Avionics 
Deputy NDE for Fluids, Thermal, and Life Support 
Orbital Sciences Corporation 
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4.0 Executive Summary 

The NESC consultation team participated in real-time troubleshooting of the Main Propulsion 
System (MPS) Engine Cut-off (ECO) sensor system failures during STS-1 14 launch countdown. 
The team assisted with External Tank (ET) thermal and ECO Point Sensor Box (PSB) circuit 
analyses, and made real-time inputs to the Space Shuttle Program (SSP) problem resolution 
teams. Several long-term recommendations resulted. One recommendation was to conduct 
cryogenic tests of the ECO sensors to validate, or disprove, the theory that variations in circuit 
impedance due to cryogenic effects on swaged connections within the sensor were the root cause 
of STS- 1 14 failures. This recommendation resulted in the initiation of a follow-on NESC 
independent technical assessment (IT A) entitled ECO Sensor Reliability Testing (NESC ITA 
number 05-098-E). 

Although anomaly troubleshooting was extensive, the root cause of the failures observed during 
the first STS-1 14 tanking test and launch scrub was not identified. The most probable causes 
include an intermittent electrical connection within the Orbiter or ET wiring from the ECO PSB 
to the sensor and return, an intermittent high resistance or open circuit in the senor itself, or a 
thermally-induced intermittent failure internal to the PSB. 
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5.0 Consultation 

The scope of this consultation was limited to real-time troubleshooting of the STS-1 14 ECO 
sensor problems. This consultation was considered a quick turnaround task and, therefore, no 
consultation plan was generated prior to this review. 
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6.0 Description of the Problem, Proposed Solutions, and Risk Assessment 

The MPS ECO system consists of point-sensors installed in the ET liquid hydrogen (LH 2 ) tank 
and the Orbiter’s liquid oxygen (L0 2 ) feedline. Point sensor electronics are designed to 
condition signals and to provide appropriate stimulation of the sensors and associated wiring and 
connectors. Refer to Figure 6.0-1 for an overall schematic. 



FOUR LH 2 ECO 
POINT SENSORS (IN ET) 


Figure 6.0-1. ECO and Liquid Level Sensors Block Diagram 

The point sensor electronics interprets a low resistance at a sensor as the presence of cryogenic 
liquid, which provides a “wet” indication to the Multiplexer/De-Multiplexer (MDM) for use by 
on-board General Purpose Computers (GPCs) and the ground Launch Processing System (LPS) 
Conversely, a high resistance is interpreted as a “dry” indication. The point sensor electronics 
include circuitry suitable for pre- flight verification of circuit function and are designed to fail 
“wet”. For example, an open circuit in the sensor, or an open or short in the signal path, will 
provide a “wet” indication to the MDM. The system is then activated and checked out during 
launch countdown and remains active during ascent. 

An ECO sensor is depicted in Figure 6.0-2. The sensor consists of a platinum wire sensing 
element mounted on an alumina Printed Wiring Board (PWB) and is encased in an aluminum 
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housing. The sensing element acts as a variable resistance which changes on exposure to 
cryogenic liquid. This resistance variation is detected by post-sensor (signal conditioning) 
electronics and is used to generate either a “wet” or “dry” indication as noted above. 


lead wire 

(FEP TEFLUM FOR LH 2 > 
(TFE FOR Ldjl 



Figure 6.0-2. ECO Sensor System Overview 

The ECO system is designed to protect the Space Shuttle Main Engines (SSMEs) from 
catastrophic failure due to propellant depletion. Flight software is coded to check for the 
presence of “wet” indications from the sensors within 8 to 12 seconds of SSME shutdown. The 
software rejects the first “dry” indication observed from any of the ECO sensors, but the 
presence of at least two more “dry” indications will result in a command to shutdown the SSMEs 
(i.e., “dry” indications from two of four “good” sensors are required for SSME shutdown). Early 
SSME shutdown would probably lead to a contingency Trans- Atlantic (TAL) abort. A failed 
“wet” indication cannot be detected. The system is designed so that LO 2 depletion should occur 
first. However, a failure “wet” indication of three of the four LH 2 sensors, coupled with an 
SSME problem that results in early LH 2 depletion, could result in catastrophic failure of a 
SSME. Failure probability is considered remote, but would almost certainly be catastrophic to 
the flight vehicle. The system architecture addresses redundancy with one point sensor box 
containing four groups of sensor conditioner circuit cards. Each card can accommodate one 
hydrogen and one oxygen sensor. Each card group has its own power converter and one sensor 
conditioner card from each group services a pair of ECO sensors (again, one hydrogen and one 
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oxygen). Wiring for each of the eight ECO sensors is split into one of two groups of sensors 
which are routed through separate Orbiter / ET monoball connections." 

During the first STS- 1 14 tanking test, LH 2 EC 0-4 failed to transition from “wet” to “dry” when 
subjected to simulated “dry” commands. During de-tank, the same sensor failed to transition to 
“dry” when the liquid level dropped below the sensor, but did transition several minutes later. 
After de-tank, LH 2 ECO-3 indicated “wet” for approximately 90 minutes, then returned to the 
“dry” state. Extensive troubleshooting conducted after the failures did not isolate the cause and 
problems did not recur during the second STS-1 14 tanking test. During the first STS-1 14 launch 
attempt, ECO-2 failed to transition from “wet” to “dry” when the “dry-when-wet” state was 
commanded. This sensor remained “wet” until about three hours into de-tank boil-off when it 
transitioned to “dry.” 

The NESC consultation team participated in subsequent troubleshooting. 
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7.0 Data Analysis 

The NESC consultation team did not conduct independent test or analysis during the course of 
this consultation, but participated with the SSP teams in real-time troubleshooting of the STS- 
1 14 ECO sensor problems. Section 8.0 provides the team’s findings, observations, and 
recommendations. 

Troubleshooting after the first launch attempt was impaired by the inability to recreate the 
anomalous system behavior. It only occurred when the vehicle was fueled and ready for launch. 
While a change was implemented to facilitate troubleshooting by swapping ECO sensors 
between two PSB signal conditioner channels, the overall effort was constrained by a 
programmatic requirement to maintain the vehicle in a launch-ready configuration. 
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8.0 Findings, Observations and Recommendations 

8.1 Findings 

F-l. STS- 1 14 troubleshooting did not isolate the root cause of the ECO sensor failures. 

F-2. The SSP does not test ECO sensors prior to launch countdown in LEE at -423 deg F, but 
tests them in liquid nitrogen (LN 2 ) at -320 deg F. 

F-3. Isolation and continuity checks of PSB connectors are not routinely performed. 

F-4. PSB transient suppressor electronics are not subjected to routine functional tests. 

F-5. An Electromagnetic Interference (EMI) induced failure observed during testing of 

modified PSB electronics was reproduced at the NASA Shuttle Logistics Depot (NSLD). 
The failure was traced to circuit differences between a flight unit and the modified PSB 
under test, which was removed from the Shuttle Avionics Integration Laboratory (SAIL). 
It was initiated by non-standard sensor test resistance values and has been explained by 
circuit analysis and testing. 

F-6. PSB electronics were subjected to stability testing in the past for several hours with no 
heat-sinking and no temperature monitoring. While current practice is to test the boxes 
on a heat-sink, latent effects of the earlier testing are unknown. 

F-7. PSB electronics have been the subject of a number of workmanship issues including poor 
solder joints, lack of solder on some connections, and discrepant flex ribbon assemblies 
(the flex ribbon assemblies have been especially problematic, according to NSLD 
personnel). Extent of remaining workmanship problems is unknown. 

8.2 Observations 

0-1. While the root cause of the STS-1 14 failure was not identified, two probable causes 
surfaced including an intermittent open somewhere in the wiring from the PSB to the 
sensors and return, an intermittent high resistance or open of the sensor itself, or a 
thermally-induced intermittent failure internal to the PSB. To minimize flight risk, the 
SSP implemented a set of real-time screens during launch countdown to help detect and 
isolate a failure. The SSP determined, and the NESC concurred, that launch with a 
single failed ECO sensor was acceptable providing the problem could be isolated to the 
one sensor/electronics channel which failed during the first STS-1 14 launch attempt. 
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Sensor 2 and 4 PSB inputs were switched to facilitate real-time troubleshooting and 
problem isolation during launch countdown, but the failure did not recur. 

0 - 2 . There are a number of open questions pertaining to the ECO sensor. The sensors are not 
screened at LH 2 temperatures prior to installation as noted in Finding F-l. This may 
affect their reliability. The design incorporates swaged terminals and stainless steel 
surface passivation which may also contribute to intermittent cold opens. 

0-3. Since multiple power grounds enter the PSBs, they must be shown to be isolated through 
routine continuity and isolation tests performed at the interface connectors. The chassis 
continuity connections need to be verified, as do the transient suppressors installed 
across the sensor lines and 28-volt bus in the power converters. 

0 - 4 . Transient suppressors used in this design may be incorrectly sized. Review of the PSB 
power converter schematic (1500067, revision T) shows the Transient Voltage 
Suppressors (TVS) to be back-to-back 1N561 Is. According to the data sheet for this 
part, it begins to zener at 43.7 volts and clamps at 63.5 volts at peak surge current. The 
IC1 and IC2 op-amps are 741s and are directly across the unfiltered 28-volt supply in 
the drawings reviewed (1500062-005 revision D). According to the data sheet, the 
LM741 absolute maximum Vcc rating is +22 volts, or 44 volts total. It appears that 
input voltage surges, particularly strong ones, may be able to overvoltage stress the op- 
amps if these schematics are correct. 

0 - 5 . A problem with loose card guides noted during troubleshooting of the STS-1 14 ECO 
problems has not been fully resolved. The guides in at least one PSB were re-glued 
(S/N 110). However, when the unit was disassembled due to a card guide lug short 
circuit, the guides were found to have again de-bonded. The guides are necessary to 
help dissipate a relatively high heat load (over 100 watts of waste heat) and are a 
potential source of internal contamination when de-bonded due to the potential for 
liberation of copper beads incorporated in the epoxy adhesive to improve thermal 
conductivity. 

0 - 6 . There are a number of lingering issues with parts used in the PSBs. Government- 

Industry Data Exchange Program (GIDEP) Alert BZ-A-80-01D covered many Fairchild 
Semiconductor part numbers and Lot Date Codes (LDC). Parts considered suspect 
during the STS-1 14 troubleshooting were researched (LDC 7830 2N2222), but the PSBs 
contain other transitors including 2N2907 and 2N2219, which may also be covered by 
the Alert if they were manufactured by Fairchild and are in the LDC range of concern. 

0-7. Specific issues with two of the PSB’s tested during the STS-1 14 troubleshooting have 
yet to be resolved. The PSB S/N 1 12 power converter has been noisy. This may be 
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related to power converter instability or limit cycle regulation. Age may affect capacitor 
properties that can also possibly affect stability. This should be investigated further to 
ensure high reliability. In addition, the potential for copper path damage between the 
signal conditioner card circuitry and electrical connector in PSB S/N 1 1 1 should be 
assessed. 

0-8. Test method CS1 16 has replaced CS06 as a method of testing EMI / EMC compliance. 
Although CS1 16 is a less-stringent test, unlike CS06 it applies to signal lines as well as 
power lines. The Point Sensor Box has been designed with bandwidth-limiting 
networks only on the power leads. Suspicion remains that the signal lines were never 
designed to be subjected to high input levels and CS 1 16 testing could be over-stressing 
the circuitry, particularly the transient suppressors intended to provide ESD protection. 

8.3 Recommendations 


R-l. Conduct a series of cryogenic tests to determine whether reliability issues exist with the 
ECO sensors. These tests should address issues surrounding the use of staked 
connections and passivated stainless steel surfaces. 

R-2. Modify PSB pre-flight testing and incorporate checkout of transient suppressor 
electronics and verification of multiple power grounds. 

R-3. Verify CS1 16 EMI / EMC testing does not overstress PSB input circuitry. 

R-4. Complete a review of the PSB circuit design and ensure transient suppressor electronics 
are properly sized and that all parts have been screened for any GIDEP -related issues. 

R-5. Ensure all PSBs have been screened for potential workmanship and assembly issues, 

including loose card guides and damaged flex-ribbon assemblies, and that such problems 
have been eliminated to the extent possible. 

R-6. Determine whether power-on testing of PSB electronics without a heat-sink may have 

resulted in component overstress and, if so, what components are likely to be affected and 
how failure of those components may manifest itself in flight. 

R-7. Isolate the cause of PSB S/N 112 power converter noise and inspect PSB S/N 111 for 
copper path damage between the signal conditioner card circuitry and electrical 
connector. 
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9.0 Lessons Learned 

Given the amount of time available for the STS-1 14 ECO real-time troubleshooting, the NESC’s 
contribution was limited to assembly of a knowledgeable team of experts who participated in the 
real-time troubleshooting efforts. No significant lessons-leamed were generated. 


10.0 Definition of Terms 

Corrective Actions Changes to design processes, work instructions, workmanship practices, 
training, inspections, tests, procedures, specifications, drawings, tools, 
equipment, facilities, resources, or material that result in preventing, 
minimizing, or limiting the potential for recurrence of a problem. 

Finding A conclusion based on facts established during the assessment/inspection 

by the investigating authority. 

Lessons Learned Knowledge or understanding gained by experience. The experience may 
be positive, as in a successful test or mission, or negative, as in a mishap 
or failure. A lesson must be significant in that it has real or assumed 
impact on operations; valid in that it is factually and technically correct; 
and applicable in that it identifies a specific design, process, or decision 
that reduces or limits the potential for failures and mishaps, or reinforces a 
positive result. 

Observation A factor, event, or circumstance identified during the 

assessment/inspection that did not contribute to the problem, but if left 
uncorrected has the potential to cause a mishap, injury, or increase the 
severity should a mishap occur. 

Passivation The process of making a material "passive" in relation to another material 

prior to using the materials together. In the context of corrosion, 
passivation is the spontaneous formation of a hard surface film which 
inhibits further corrosion. Under normal conditions of pH and oxygen 
concentration, passivation is seen in such materials as aluminum, stainless 
steel, titanium, and silicon. 


Problem The subject of the technical assessment/inspection. 

Requirement An action developed by the assessment/inspection team to correct the 

cause or a deficiency identified during the investigation. The requirements 
will be used in the preparation of the corrective action plan. 
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Root Cause Along a chain of events leading to a mishap or close call, the first causal 

action or failure to act that could have been controlled systemically either 
by policy/practice/procedure or individual adherence to 
pol icy/ practice/procedure . 


11.0 Minority Report (Dissenting Opinions) 

There were no dissenting opinions during this consultation. 
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Volume II: Appendices 

A ITA/I Request Form (NESC-PR-003-FM-01) 

B List of Acronyms 


NESC Request No. 05-045-E 


m 

NASA Engineering and Safety Center 
Technical Consultation Report 

Document #: 

RP-05-125 

Version: 

1.0 

Title: 

STS-114 Engine Cut-off Sensor Anomaly 
Technical Consultation Report 

Page #: 

18 of 23 


Appendix A. ITA/I Request Form (NESC-PR-003-FM-01) 
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NASA Engineering and Safety Center 
Request Form 


Submit this IT A/I Request, with associated artifacts attached, to: lirbexCCSeC ffl.nasa.gQV. or to 
NRB Executive Secretary. M/S 105, NASA Langley Research Center. Hampton. VA 23681 


Section 1: NESC Review’ Board (NRB) Executive Secretary Record of Receipt 


Received (mm/dd/yyyy h:mm ant pm) 
7/14/2005 12:00 AM 


Status: New 


Reference #: 05-045-E 


Initiator Name: John Murat ore 


E-mail: 

j oh n . f. ni uralo re @n as a. go v 


Center: JSC 


Phone: (281 )-483-4467, Ext . 


Mail Stop: MS 


Short Title: STS- 1 14 Engine Cut-off Sensor Anomaly 


Description: Support the Space Shuttle Program efforts to resolve the STS- 1 14 Engine Cut-Off (ECO) sensor 
anomaly Three people from the NESC are actively engaged: Bob Kiekak working the ECO sensor electronics 
analysis, Gene Ungar, participating with the ET thermal modeling team mid Steve Rickman enlisted by Gene to 
participate on the sensor electronics thermal analysis team. 


Source (e g email, phone call, posted on weh): Verbal to Ralph Roc 


't ype of Request: Consultation 


Proposed Need Date: 


Date forwarded to Systems Engineering Office (SEO): (mmddvyvv h:mm am/pm): 


Section 2: Systems Engineering Office Screening 


Section 2.1 Potential IT A/I Identification 


Received by SEO: (mm dd vyyy h mni am.pm): 7 14 2005 12:00 AM 


1 


Potential 1TA/I candidate? MVes LJ No 


Assigned Initial F, valuator (IE): Tim Wilson 


Date assigned (mm dd yvyv): 7 14-2005 


Due date for ITA'I Screening (mm/dd ■•yyyy): 


Section 2,2 Non-IT A/I Action 


Requires additional NESC action (non-ITA/D? I I Yes □ No 


If ves: 


Description of action: 


Actionee: 


Is follow-up required? | |Yes No If yes: Due Date: 


F o llow-up status d ate : 


If no: 


NESC Director Concurrence (signature): 


Recjuest^ 


closure date: 


NESC Request Form Page 1 of 3 

NESC-PR-003-FM-01 . v 1 .0 
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Section 3: Initial Evaluation 
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